Field of the Invention
The present invention relates to a technique for performing focus detection by a phase difference detection method using image signals from an image sensor that has pixels for focus detection on the imaging plane.
Description of the Related Art
Conventionally, as focus detection methods for performing focus control in image capturing apparatuses, various methods have been proposed, such as a phase difference detection method that uses focus detection elements, and a contrast method that uses the contrast component of an image from an image sensor. Also, there is known to be a technique for performing focus detection by a phase difference detection method at the same time as image capturing, by adopting a configuration in which pixels in an image sensor receive light from different pupil regions of an imaging lens.
Japanese Patent Laid-Open No. 2001-083407 discloses an image sensor in which a luminous beam converged by one microlens in one pixel is photoelectrically converted using divided photodiodes (hereinafter, PDs), and thereby each PD receives light from a different pupil region of an imaging lens. Focus detection for the imaging lens is then performed by comparing output of the two PDs.
Also, Japanese Patent Laid-Open No. 2014-182360 discloses a technique for reading out the electric charges of a portion of the PDs under one microlens, then reading out the addition value of the electric charges of all the PDs under the same microlens, and thereby generating the value of the other photoelectric conversion portion from the difference between those values. Accordingly, focus detection is performed while maintaining the characteristics of image signals for image capturing.
However, the conventional techniques disclosed in Japanese Patent Laid-Open No. 2001-083407 and Japanese Patent Laid-Open No. 2014-182360 above have the following problem. For example, in the case where a PD is divided into two under one microlens, the pixel signal corresponding to the electric charge of one PD is assumed to be an A image, and the pixel signal corresponding to the electric charge of the other PD is assumed to be a B image. Also, the addition value of the signals of all the PDs is assumed to be an A+B image.
In focus detection, the A image is read out, the A+B image is then read out, and the B image is generated by subtracting the A image from the A+B image, in order to perform calculation of focus detection. It is also possible to independently read out the A image and the B image and perform calculation for focus detection, but the A+B image is also required in order to generate an image. In that case, if the A image and the B image, both of which have random noise, are added and the A+B image is generated, the random noise amount of the A+B image increase due to the addition. If the random noise amount increases, the image quality deteriorates, and thus a configuration is adopted here in which instead of reading out the A image and the B image, the A image and the A+B image are independently read out. In calculation of focus detection, an A image row includes pixels arranged in the horizontal direction, a B image row includes corresponding pixels arranged in the horizontal direction, the B image row is subtracted from the A image row while shifting the A image row and the B image row in the horizontal direction, the absolute values are then calculated, and the absolute values are added to obtain a correlation value of the A image and the B image (correlation calculation). A defocus amount is calculated from the obtained correlation value.
For example, in FIG. 26, a PD for A image is denoted by 1101, and a PD for B image is denoted by 1102, which are under one microlens 1100. The A images of the row of pixels arranged in the horizontal direction are denoted by A0 to A7, and the B images of the row of pixels arranged in the horizontal direction are denoted by B0 to B7. Data generated by the above processing is subjected to correlation calculation for the A image row and the B image row. As shown in FIG. 27, when the shift amount is −2, the B image row is shifted relative to the A image row by two pixels to the left, and the absolute value of the value obtained by subtracting the corresponding pixel value after the shift is calculated. In this case, correlation values P(0) to P(3) of the pixels are calculated as:P(0)=|A2−B4|,P(1)=|A3−B5|,P(2)=|A4-B6|, andP(3)=|A5−B7|,and a correlation value P[−2] when the shift amount is −2 is calculated by adding the correlation values P(0) to P(3) as follows:P[−2]=P(0)+P(1)+P(2)+P(3)
In a similar manner, correlation values in the case of other shift amounts are also calculated as shown in FIG. 27. At this time, A image signals corresponding to the electric charges of the PDs in a k-th row are assumed to be S[A(k)], and (A+B) image signals are assumed to be S[(A+B)(k)]. Also, random noise that is caused by a readout circuit and is superimposed on pixel data when the A image is read out is assumed to be N[A(k)], and random noise that is caused by the readout circuit and is superimposed on pixel data when the (A+B) image is readout is assumed to be N[(A+B)(k)]. When the B image is generated in accordance with the above description, the following expression holds true.
                              B          ⁢                                          ⁢          image                =                ⁢                                            (                              A                +                B                            )                        ⁢                                                  ⁢            image                    -                      A            ⁢                                                  ⁢            image                                                  =                ⁢                              (                                          S                ⁡                                  [                                                            (                                              A                        +                        B                                            )                                        ⁢                                          (                      k                      )                                                        ]                                            +                              N                ⁡                                  [                                                            (                                              A                        +                        B                                            )                                        ⁢                                          (                      k                      )                                                        ]                                                      )                    -                      (                                          S                ⁡                                  [                                      A                    ⁡                                          (                      k                      )                                                        ]                                            +                              N                ⁡                                  [                                      A                    ⁡                                          (                      k                      )                                                        ]                                                      )                                                  =                ⁢                              S            ⁡                          [                                                (                                      A                    +                    B                                    )                                ⁢                                  (                  k                  )                                            ]                                -                      S            ⁡                          [                              A                ⁡                                  (                  k                  )                                            ]                                +                      N            ⁡                          [                                                (                                      A                    +                    B                                    )                                ⁢                                  (                  k                  )                                            ]                                -                      N            ⁡                          [                              A                ⁡                                  (                  k                  )                                            ]                                          
In correlation calculation for one row when the shift amount is S (S is not 0), correlation calculation for the A image of an m-th row and the B image of an n-th row is performed, and a correlation value P(h) at this time is as follows.
                              P          ⁡                      (            h            )                          =                ⁢                                                      A              ⁢                                                          ⁢              image                        -                          B              ⁢                                                          ⁢              image                                                                        =                ⁢                                                                                                                  (                                                                  S                        ⁡                                                  [                                                      A                            ⁡                                                          (                              m                              )                                                                                ]                                                                    +                                              N                        ⁡                                                  [                                                      A                            ⁡                                                          (                              m                              )                                                                                ]                                                                                      )                                    -                                      (                                                                  S                        ⁡                                                  [                                                                                    (                                                              A                                +                                B                                                            )                                                        ⁢                                                          (                              n                              )                                                                                ]                                                                    -                                                                                                                                                                                                          S                      ⁡                                              [                                                  A                          ⁡                                                      (                            n                            )                                                                          ]                                                              +                                          N                      ⁡                                              [                                                                              (                                                          A                              +                              B                                                        )                                                    ⁢                                                      (                            n                            )                                                                          ]                                                              -                                          N                      ⁡                                              [                                                  A                          ⁡                                                      (                            n                            )                                                                          ]                                                                              )                                                                                                            =                ⁢                                                                                                                  S                    ⁡                                          [                                              A                        ⁡                                                  (                          m                          )                                                                    ]                                                        +                                      S                    ⁡                                          [                                              A                        ⁡                                                  (                          n                          )                                                                    ]                                                        -                                      S                    ⁡                                          [                                                                        (                                                      A                            +                            B                                                    )                                                ⁢                                                  (                          n                          )                                                                    ]                                                        +                                                                                                                                              N                    ⁡                                          [                                              A                        ⁡                                                  (                          m                          )                                                                    ]                                                        +                                      N                    ⁡                                          [                                              A                        ⁡                                                  (                          n                          )                                                                    ]                                                        -                                      N                    ⁡                                          [                                                                        (                                                      A                            +                            B                                                    )                                                ⁢                                                  (                          n                          )                                                                    ]                                                                                                                              Here, the higher the correlation between the A image and the B image is, the smaller this correlation value P(h) becomes.
Also, in correlation calculation for one row when the shift amount is 0, correlation calculation for the A image of the m-th row and the B image of the m-th row is performed, and the correlation value P(h) at this time is as follows.
                              P          ⁡                      (            h            )                          =                ⁢                                                      A              ⁢                                                          ⁢              image                        -                          B              ⁢                                                          ⁢              image                                                                        =                ⁢                                                                                                                  (                                                                  S                        ⁡                                                  [                                                      A                            ⁡                                                          (                              m                              )                                                                                ]                                                                    +                                              N                        ⁡                                                  [                                                      A                            ⁡                                                          (                              m                              )                                                                                ]                                                                                      )                                    -                                      (                                                                  S                        ⁡                                                  [                                                                                    (                                                              A                                +                                B                                                            )                                                        ⁢                                                          (                              m                              )                                                                                ]                                                                    -                                                                                                                                                                                                          S                      ⁡                                              [                                                  A                          ⁡                                                      (                            m                            )                                                                          ]                                                              +                                          N                      ⁡                                              [                                                                              (                                                          A                              +                              B                                                        )                                                    ⁢                                                      (                            m                            )                                                                          ]                                                              -                                          N                      ⁡                                              [                                                  A                          ⁡                                                      (                            m                            )                                                                          ]                                                                              )                                                                                                            =                ⁢                                                                                                                  S                    ⁡                                          [                                              A                        ⁡                                                  (                          m                          )                                                                    ]                                                        +                                      S                    ⁡                                          [                                              A                        ⁡                                                  (                          m                          )                                                                    ]                                                        -                                      S                    ⁡                                          [                                                                        (                                                      A                            +                            B                                                    )                                                ⁢                                                  (                          m                          )                                                                    ]                                                        +                                                                                                                                              N                    ⁡                                          [                                              A                        ⁡                                                  (                          m                          )                                                                    ]                                                        +                                      N                    ⁡                                          [                                              A                        ⁡                                                  (                          m                          )                                                                    ]                                                        -                                      N                    ⁡                                          [                                                                        (                                                      A                            +                            B                                                    )                                                ⁢                                                  (                          m                          )                                                                    ]                                                                                                                              
Here, the amount of random noise superimposed on the correlation value when the shift amount is S (S is not 0) and the amount of random noise superimposed on the correlation value when the shift amount is 0 are as follows.When shift amount=S:Noise(S)=N[A(m)]+N[A(n)]−N[(A+B)(n)], andWhen shift amount=0:Noise(0)=N[A(m)]+N[A(m)]−N[(A+B)(m)].
N[A(m)], N[A(n)] and N[(A+B) (n)] are random noise not correlated with each other, and thus Noise(S) will be a nearly constant value when the shift amount is not 0, as shown in FIG. 28A. On the other hand, regarding Noise(0), although N[A(m)] and N[(A+B)(m)] are random noise not correlated with each other, N[A(m)] is doubled (N[A(m)]*2) and is larger than Noise(S). Therefore, as shown in FIG. 28A, only when the shift amount is 0, the correlation amount increases due to random noise.
As described above, in the case of generating the B image from the A+B image and the A image and performing correlation calculation, a random noise amount obtained by inverting the sign of a random noise amount superimposed on the A image is superimposed on the B image, and thereby when the shift amount=0, the noise amount of N[A(m)]*2 is always superimposed on the correlation value. Accordingly, a local peak is generated.
In cases of shooting an object for which the contrast between the A image and the B image is low and shooting in a low luminance environment, the correlation value P(h) decreases, but in this case, the noise amount expressed by N[A(m)]*2 becomes dominant. Without noise influence, the correlation value P(h) is smallest (H point in FIG. 28B) when the shift amount is 0 while the imaging optical system is in focus, and focus detection is appropriately performed by detecting this point. However, when the noise amount of N[A(m)]*2 is dominant, as shown in FIG. 28C, a result will be obtained that the correlation value P(h) increases (correlation is low) (I point), when the shift amount is 0. As a result, there has been a problem in that a plurality of points where the correlation value P(h) is lowest are generated (J point, K point), those points are recognized as a shift amount of an in-focus position, and as a result, erroneous ranging and hunting occur.